Image processing apparatus, image processing method, and recording medium recording a program

ABSTRACT

An image processing apparatus includes an obtainment section to obtain a face image; an area specifying section to specify a set of a plurality of corresponding areas in the face image obtained by the obtainment section; and a correction section to generate a face image in which one area of the plurality of corresponding areas of the face image is used as a reference to correct another area.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to an image processing apparatus, an imageprocessing method, and a recording medium recording a program.

2. Description of Related Art

The technique of deforming a face image by expanding or reducing a partof the face image for an automatic photographing apparatus outputtingthe face image produced by imaging the face of a subject as a print wasproposed (see, for example, Japanese Patent Application Laid-OpenPublication No. 2000-267156).

However, the deforming of a face image by simply expanding or reducing apart thereof as disclosed in Japanese Patent Application Laid-OpenPublication No. 2000-267156 has a problem of producing an unnaturalimage owing to the processed part of the image being conspicuous.

BRIEF SUMMARY OF THE INVENTION

Accordingly, it is an object of the present invention to perform morenatural image processing to a face image.

According to an embodiment of the present invention, there is providedan image processing apparatus, including: an obtainment section toobtain a face image; an area specifying section to specify a set of aplurality of corresponding areas in the face image obtained by theobtainment section; and a correction section to generate a face image inwhich one area of the plurality of corresponding areas of the face imageis used as a reference to correct another area.

According to an embodiment of the present invention, there is providedan image processing apparatus, including: an obtainment section toobtain a face image; a face constituting part specifying section tospecify a set of a plurality of corresponding face constituting parts inthe face image obtained by the obtainment section; and a correctionsection to generate a face image in which one face constituting part ofthe plurality of corresponding face constituting parts of the face imageis used as a reference to correct another face constituting part.

According to an embodiment of the present invention, there is providedan image processing method for making an image processing apparatusexecute the pieces of processing including: obtaining a face image;specifying a set of a plurality of corresponding areas in the obtainedface image; and generating a face image in which one are of theplurality of specified corresponding areas of the face image is used asa reference to correct another area.

According to an embodiment of the present invention, there is providedan image processing method for making an image processing apparatusexecute the pieces of processing including: obtaining a face image;specifying a set of a plurality of corresponding face constituting partsin the obtained face image; and generating a face image in which oneface constituting part of the plurality of specified corresponding faceconstituting parts of the face image is used as a reference to correctanother face constituting part.

According to an embodiment of the present invention, there is provided arecording medium recording a program for making a computer installed inan image processing apparatus execute the functions including: anobtainment section to obtain a face image; an area specifying section tospecify a set of a plurality of corresponding areas in the face imageobtained by the obtainment section; and a correction section to generatea face image in which one area of the plurality of corresponding areasof the face image specified by the area specifying section is used as areference to correct another area.

According to an embodiment of the present invention, there is provided arecording medium recording a program for making a computer installed inan image processing apparatus execute the functions including: anobtainment section to obtain a face image; a face constituting partspecifying section to specify a set of a plurality of corresponding faceconstituting parts in the face image obtained by the obtainment section;and a correction section to generate a face image in which one faceconstituting part of the plurality of corresponding face constitutingparts of the face image is used as a reference to correct another faceconstituting part.

BRIEF DESCRIPTION OF THE SEVERAL VIEWS OF THE DRAWINGS

FIG. 1 is a diagram showing an example of an imaging apparatusfunctioning as the image processing apparatus of an embodiment to whichthe present invention is applied;

FIG. 2 is a flow chart showing an example of the flow of imagingprocessing;

FIG. 3 is a flow chart showing an example of the flow of correctionprocessing;

FIG. 4 is a flow chart showing an example of the flow of area specifyingprocessing;

FIG. 5 is a flow chart showing an example of the flow of correctionprocessing of an eye;

FIG. 6 is a flow chart showing an example of the flow of correctionprocessing of a corner of a mouth;

FIG. 7 is a view showing an example of a display based on the image dataof a face image on a display section;

FIG. 8 is a view showing an example of a display based on the image dataof the face image after a correction;

FIG. 9 is a diagram showing an example of a modification of the imagingapparatus;

FIG. 10 is a flow chart of an example of the flow of correctionprocessing of the modification; and

FIG. 11 is a flow chart showing the continuation of the flow of FIG. 10.

DETAILED DESCRIPTION OF THE INVENTION

In the following, an embodiment of the present invention will bedescribed with reference to the accompanying drawings. However, thescope of the invention is not limited to the shown examples.

FIG. 1 is a diagram showing an imaging apparatus 100 functioning as theimage processing apparatus of an embodiment to which the presentinvention is applied.

The imaging apparatus 100 functioning as the image processing apparatusobtains the image data of a face image, specifies a plurality of areason both sides of a center line extending into a predetermined directionat almost the central part of the obtained face image as a reference,and uses one of the plurality of specified areas of the face image as areference to correct another area.

To put it concretely, as shown in FIG. 1, the imaging apparatus 100includes a lens section 1, an electronic imaging section 2, an imagingcontrol section 3, an image data generating section 4, an image memory5, a recording medium 6, a display control section 7, a display section8, an operation input section 9, a central processing unit (CPU) 10, anda face image processing section 11.

Moreover, the imaging control section 3 and the CPU 10 are designed as,for example, a custom large scale integrated circuit (LSI) 1A togetherwith the face image processing section 11.

The lens section 1 is composed of a plurality of lenses and includes azoom lens, a focus lens, and the like.

Moreover, the lens section 1 may include a zoom drive section for movingthe zoom lens into the optical axis direction thereof at the time ofimaging a subject (see, for example, a face F shown in FIG. 7), afocusing drive section for moving the focus lens into the optical axisdirection thereof at the imaging time, and the like, although theillustration of those sections are omitted.

The electronic imaging section 2 images a subject to generate the imagedata thereof. The electronic imaging section 2 includes an image sensor,such as a charge coupled device (CCD) or a complementary metal-oxidesemiconductor (CMOS), and converts an optical image that has passedthrough various lenses of the lens section 1 into a two-dimensionalimage signal.

The imaging control section 3 includes a timing generator, a driver, andthe like, although their illustration is omitted. The imaging controlsection 3 drives the electronic imaging section 2 to scan the electronicimaging section 2 with the timing generator and the driver, and makesthe electronic imaging section 2 convert an optical image into atwo-dimensional image signal every predetermined period. Then, theimaging control section 3 reads out the image frame of every screen fromthe imaging area of the electronic imaging section 2 to output theread-out image frame to the image data generating section 4.

Moreover, the imaging control section 3 controls the imaging conditionsof a subject S, such as the settings of automatic focusing (AF)processing, automatic exposure (AE) processing, and automatic whitebalancing (AWB) processing, and adjusts them.

FIG. 7 is an example of displaying the image data of a face image in thedisplay section 8.

The image data generating section 4 suitably performs the gainadjustment of each of the color components of red (R), green (G), andblue (B) of the signal having an analog value of the image frametransferred from the electronic imaging section 2. After that, the imagedata generating section 4 performs the sample-and-hold action of thegain-adjusted color components by the use of a sample-and-hold circuit(the illustration of which is omitted) and converts the sample-heldsignal into digital data by the use of an analog-to-digital (A/D)converter (the illustration of which is omitted). The image datagenerating section 4 further performs color process processing includingpixel interpolation processing and 7 correction processing with a colorprocess circuit (the illustration of which is omitted), and after that,the image data generating section 4 generates a luminance signal Y andchrominance difference signals Cb and Cr (YUV data) having digitalvalues.

The luminance signal Y and the chrominance difference signals Cb and Croutput from the color process circuit are transferred to the imagememory 5, used as a buffer memory, by a direct memory access (DMA)transfer through a not-shown DMA controller.

The image memory 5 is composed of, for example, a dynamic random accessmemory (DRAM), and temporarily stores data and the like that areprocessed by the CPU 10, the face image processing section 11, and thelike.

The recording medium 6 is composed of, for example, a nonvolatile memory(such as a flash memory), and records the recording image data of animaged image coded by a Joint Photographic Experts Group (JPEG)compression section (the illustration of which is omitted) of the imageprocessing section 11.

The display control section 7 performs the control of reading outdisplaying image data temporarily stored in the image memory 5 and imagedata recorded in the recording medium 6 to make the display section 8display the read image data therein.

To put it concretely, the display control section 7 includes a videorandom access memory (VRAM), a VRAM controller, a digital video encoder,and the like. Then, the digital video encoder periodically reads out theluminance signal Y and the chrominance difference signals Cb and Crwhich have been read-out from the image memory 5 and are stored in theVRAM (the illustration of which is omitted) from the VRAM through theVRAM controller under the control of the CPU 10. The digital videoencoder then generates a video signal on the basis of these pieces ofdata to output the generated video signal to the display section 8.

Moreover, the display control section 7 makes the display section 8display a correction result by the correction section 11 c (describedbelow) of the face image processing section 11.

Furthermore, the display control section 7 makes the display section 8display the correction result and further performs confirming processingfor enabling a user to confirm whether to save the displayed correctionresult or not. As the confirming processing, the display control section7, for example, makes the display section 8 display a saving proprietyselecting display for selecting whether to save the displayed correctionresult into the face image or not.

Then, a user performs an input operation through the determinationbutton 9 c or the cancel button 9 d (described below) of the operationinput section 9 as a selection input of whether to save the correctionresult or not in the saving propriety selecting processing.

The display section 8 is, for example, a liquid crystal displayapparatus and displays an image imaged by the electronic imaging section2 and the like on the display screen thereof on the basis of a videosignal from the display control section 7. To put it concretely, thedisplay section 8 displays a live view image on the basis of a pluralityof image frames generated by imaging an subject by means of the lenssection 1, the electronic imaging section 2, and the imaging controlsection 3, or a face image imaged as the main imaged image in an imagingmode.

The operation input section 9 is an input section for performingpredetermined operations of the imaging apparatus 100. To put itconcretely, the operation input section 9 includes a shutter button 9 afor an photographing instruction of a subject, a mode button 9 b for aselection instruction for selecting an imaging mode, a function, and thelike on a menu screen, a zoom button (the illustration of which isomitted) for an adjustment instruction of a zoom quantity, thedetermination button 9 c and the cancel button 9 d for determiningwhether to perform a correction or not, and the like. The operationinput section 9 outputs a predetermined operation signal in response toan operation of one of those buttons to the CPU 10.

The CPU 10 controls each section of the imaging apparatus 100. To put itconcretely, the CPU 10 performs various control operations in conformitywith various processing programs (the illustration of which is omitted)for the imaging apparatus 100.

The face image processing section 11 includes a specification section 11a, a comparison section 11 b, the correction section 11 c, a correctionreference determining section 11 d, and a correction executiondetermining section 11 e.

The specification section 11 a detects a face image from image dataimaged and obtained in cooperation with the lens section 1, theelectronic imaging section 2, and the imaging control section 3. Variousmethods, such as the detection based on a comparison of a face imagewith a predetermined template and the detection based on a contourextraction of the face image, can be adopted for the detection of theface image.

Moreover, the specification section 11 a specifies a plurality of areasby the use of the center line, as a reference, extending into apredetermined direction at almost the central part of a face imageobtained by means of the lens section 1, the electronic imaging section2, and the imaging control section 3.

To put it concretely, the specification section 11 a detects the faceconstituting parts corresponding to the two eyes in the image frame of aface image. Moreover, the specification section 11 a detects thecoordinates of the center of each pupil of the two eyes (for example thecoordinates (Xa, Ya) and the coordinates (Xb, Yb) shown in FIG. 8). Thecoordinates here mean positional information to the origin (for examplethe origin O shown in FIG. 8) which positional information is expressedon the basis of predetermined two directions (for example the Xdirection and the Y direction shown in FIG. 8) perpendicular to eachother to the imaging area, which origin is situated at a predeterminedposition of the image data of the face image. After having detected thecoordinates of the centers of the respective pupils of the two eyes, thespecification section 11 a calculates the locus of the line segment (forexample the line segment C shown in FIG. 8) passing through theintermediate point of the coordinates of the centers of he respectivepupils of the detected two eyes and extending almost perpendicular tothe line segment connecting the coordinates of the centers of therespective pupils of the detected two eyes. The specification section 11a sets the line segment as the center line extending into thepredetermined direction at almost the central part of the face image.The specification section 11 a specifies the two areas of the dividedface image (for example the areas F1 and F2 shown in FIG. 8) by usingthe line segment as a reference.

The “set” described in the following indicates two common configurationsor two areas having common configurations. For example, faceconstituting parts such as two eyes (for example the left eye El and theright eye Er shown in FIG. 7) and two corners of a mouth (for examplethe left corner Ml and the right corner Mr shown in FIG. 7), and areasof a face image which areas include one or more of these constitutingparts can be cited.

Moreover, the specification section 11 a specifies a set of a pluralityof corresponding face constituting parts from an obtained face image asa set of a plurality of corresponding areas.

For example, the specification section 11 a specifies both eyes (forexample, the left eye El and the right eye Er shown in FIG. 7) as theset of the plurality of corresponding face constituting parts in a faceimage. The specification section 11 a specifies both eyes (two eyes) by,for example, specifying the pupils or whites of the eyes existing in aface image. In this case, the specification section 11 a specifies theeyes on the basis of the differences of the pixel values, thebrightness, and the like between those of the pupils and the whites ofthe eyes and those of the peripheral areas of the face image.

Moreover, the specification section 11 a obtains the informationpertaining to the sizes of eyes. For example, the specification section11 a obtains the numbers of pixels of each of the respective pupils oftwo eyes which pixels are successive along the Y direction.

Moreover, the specification section 11 a specifies the two corners ofthe mouth (for example the left corner Ml and the right corner Mr shownin FIG. 7) existing in a face image as a set of a plurality ofcorresponding face constituting parts in the face image. For example,the specification section 11 a specifies the corners of the mouth on thebasis of the differences of the pixel values, the brightness, and thelike between the areas of a face image.

It has been supposed for convenience' sake that the eye situated on theobserver's left side is expressed as the left eye El and the eyesituated on the observer's right side is expressed as the right eye Erbetween the two eyes existing in the face image shown in FIG. 7. Theleft eye El is the right eye of the subject, and the right eye Er is theleft eye of the subject.

Moreover, it has been supposed that the corner of the mouth situated onthe observer's left side is expressed as the left corner Ml and thecorner of the mouth situated on the observer's right side is expressedas the right corner Mr between the two corners of the mouth existing inthe face image shown in FIG. 7. The left corner Ml is the right cornerof the mouth of the subject, and the right corner Mr is the left cornerof the mouth of the subject.

Moreover, the specification section 11 a obtains the informationpertaining to the positions of the two corners of a mouth. In the caseof the face image shown in FIG. 8, the specification section 11 aobtains the coordinates (Y1 and Y2) of the two corners of the mouthwhich coordinates pertain to the Y direction.

Moreover, the specification section 11 a obtains the informationpertaining to the position of the lowermost end of a mouth as theinformation pertaining to the positions of the two corners of the mouth.“The most lower end portion of a mouth” here indicates the farther endof the mouth from the eyes between the ends of the mouth with respect tothe direction of a center line (for example the line segment C shown inFIG. 8) extending in the predetermined direction at almost the centralpart of a face image. In the case of the face image shown in FIG. 8, thespecification section 11 a obtains the coordinate (Y3) in the Ydirection pertaining to the position of the lowermost end of the mouthshown in FIG. 8.

The comparison section 11 b compares each of a set of two correspondingface constituting parts existing in a face image and being specified bythe specification section 11 a, and outputs a judgment result based on apredetermined condition.

For example, the comparison section 11 b compares the sizes of two eyesspecified by the specification section 11 a as a set of a plurality ofcorresponding face constituting parts. To put it concretely, thecomparison section 11 b, for example, compares the number of pixels ofthe respective pupils of the two eyes which pixels are successive alongthe Y direction and have been obtained by the specification section 11a. Then, the comparison section 11 b specifies the eye the number ofpixels of which is relatively larger. Moreover, the comparison section11 b obtains the ratio of the sizes of two eyes. For example, thecomparison section 11 b calculates the ratio of the numbers of thepixels of the respective pupils of the two eyes which pixels aresuccessive along the Y direction, and the comparison section 11 b setsthe calculated ratio as the ratio of the sizes of the two eyes.

Although the present embodiment compares and specifies the sizes of eyeson the basis of the numbers of the pixels of their pupils which pixelsare successive along the Y direction, the sizes of eyes may be comparesby other methods. For example, the comparison may be performed on thebasis of the maximum values of the numbers of the pixels of therespective whites of each of the two eyes which pixels are successivealong the X direction; the sizes of eyes may synthetically be comparedand judged on the basis of both of the numbers of the pixels of theirpupils which pixels are successive along the Y direction and the numbersof the pixels of the respective whites of the two eyes which pixels aresuccessive along the X direction; the numbers of pixels constituting therespective pupils and the whites of the two eyes may be compared witheach other.

Moreover, the comparison section 11 b compares the positions of the twocorners of a mouth specified by the specification section 11 a as a setof a plurality of corresponding face constituting parts. For example,the comparison section 11 b calculates the positions of the coordinatesof the respective two corners of a mouth which coordinates concern the Ydirection as the positions of the corners of the mouth which positionsof the corners have been obtained as the corners of the mouth to theposition indicated by the coordinates in the Y direction whichcoordinates have been obtained as the information pertaining to theposition of the lowermost end of the mouth. Then, the comparison section11 b compares the calculated positions of the respective two corners ofthe mouth with each other, and judges which position of the corners ofthe mouth is higher. Then, the comparison section 11 b specifies thecorner of the mouth the position of which is higher than that of theother between the two corners of the mouth.

The correction section 11 c uses one of the two areas of the face imagespecified by the specification section 11 a as a reference to generate aface image in which the other area is corrected.

FIG. 8 shows an example of the image data of a face image after acorrection. In FIG. 8, the half-tone dot meshing of the parts of thepupils in FIG. 7 is omitted in order to clarify the correction result ofthe size of an eye. To put it concretely, the correction section 11 cuses one of the two face constituting parts in a face image as areference which parts have been specified by the specification section11 a to correct the other face constituting part. For example, thecorrection section 11 c uses one eye as a reference to correct the othereye.

In the example shown in FIG. 8, one eye (the left eye El shown in FIGS.7 and 8) is used as the reference to correct the size of the imagingarea of the other eye (the right eye Er shown in FIG. 7), and the othereye after the correction is shown as the right eye Er2. To put itconcretely, the correction section 11 c sets the eye specified by thecomparison section 11 b as the larger one as the one eye, and correctsthe size of the other eye smaller in size than that of the one eye byexpanding the size of the other eye to be almost the same as that of theone eye. Here, the correction section 11 c corrects the size of theother eye in order that the sizes of the two eyes in their widthdirections along the Y direction may be almost equal to each other onthe basis of the ratio of the numbers of the pixels of the respectivepupils of the two eyes which pixels are successive along the Y directionand have been calculated by the comparison section 11 b.

Moreover, the correction section 11 c, for example, uses one corner of amouth as a reference to correct the other corner of the mouth.

In the example shown in FIG. 8, the correction section 11 c sets thecorner of the mouth which corner has been specified by the comparisonsection 11 b as the corner the position of which is higher than that ofthe other corner (the left corner Ml shown in FIGS. 7 and 8) as the onecorner, and corrects the coordinate of the other corner of the mouth(the right corner Mr shown in FIG. 7) which coordinate concerns the Ydirection so as to be aligned with the coordinate of the one cornerwhich coordinate concerns the Y direction. In FIG. 8, the other cornerof the mouth after the correction is shown as the right corner Mr2.

The correction reference determining section 11 d determines the faceconstituting parts that are used as the references of the corrections bythe correction section 11 c on the basis of a predetermined condition.

If “the eye specified by the comparison section 11 b as larger one isset as the one eye used as the reference of correction” is, for example,used as the predetermined condition, the correction referencedetermining section 11 d receives an input of a specification result bythe comparison section 11 b and determines the one eye of the correctionreference.

Similarly, if “the corner of a mouth which corner has been specified bythe comparison section 11 b as the one the position of which is higherthan that of the other corner is set as the one corner of the mouthwhich corner is used as the reference of correction” is, for example,used as the predetermined condition, the correction referencedetermining section 11 d receives an input of a specification result anddetermines the one corner of the mouth of the correction reference.

The correction execution determining section 11 e determines whether tosave a correction result by the correction section 11 c or not on thebasis of the content of a selection input received by the determinationbutton 9 c and the cancel button 9 d of the operation input section 9.To put it concretely, if the determination button 9 c is operated to asaving propriety confirming display for selecting whether to save acorrection result displayed on the display section 8 by the displaycontrol section 7 or not, the correction execution determining section11 e outputs the image data corresponding to the displayed correctionresult to the JPEG compression section of the image processing section11. Hereby, the image data of a face image subjected to a correction bythe correction section 11 c is recorded in the recording medium 6. Onthe other hand, if the cancel button 9 d is operated to a dialog displayfor selecting whether to save the displayed correction result or not,the correction execution determining section 11 e discards the displayedcorrection result.

Next, an image processing method executed by the imaging apparatus 100will be described with reference to FIGS. 2 to 6.

FIG. 2 is a flow chart showing an example of the flow of imagingprocessing.

First, the lens section 1 operates to be focused on a subject (forexample the face F; see FIG. 7), and the electronic imaging section 2images the subject under the control of the imaging control section 3 togenerate image data (Step S1). After that, the specification section 11a of the face image processing section 11 detects a face image from theimage data obtained by the processing at Step S1 (Step S2). Then, theface image processing section 11 performs correction processing to theface image (Step S3).

Here, the correction processing will be described with reference to theflow chart of FIG. 3.

First, the specification section 11 a performs area specifyingprocessing (Step S11).

Here, the area specifying processing will be described with reference toFIG. 4.

In the area specifying processing, the specification section 11 adetects the face constituting parts corresponding to the two eyes in theimage frame of the face image (Step S31). Moreover, the specificationsection 11 a detects the coordinates of the centers of the respectivepupils of the two eyes (Step S32). Then, the specification section 11 acalculates the locus of a first line segment (for example the linesegment C shown in FIG. 8) almost perpendicular to a second line segmentat the intermediate point thereof which second line segment connects thecoordinates of the centers of the two pupils to each other. Thespecification section 11 a specifies the first line segment as thecenter line extending into the predetermined direction at almost thecentral part of the face image (Step S33). The specification section 11a divides the face image into left and right areas (for example theareas F1 and F2 shown in FIG. 8) by using the center line as a reference(Step S34). With that, the area specifying processing ends.

After the area specifying processing, the specification section 11 afurther specifies face constituting parts (for example two eyes and thetwo corners of a mouth) that exist in each of the two left and rightareas in common (Step S12). The specification section 11 a specifieseyes and a mouth on the basis of, for example, differences of pixelvalues, brightness, and the like between the eyes and the mouth and theperipheral areas of the face image. Moreover, the specification section11 a obtains the coordinates indicating the information pertaining tothe sizes of the two eyes and the coordinates indicating the informationpertaining to the positions of the two corners of the mouth (Step S13).The specification section 11 a, for example, obtains the numbers of thepixels of the respective pupils of the two eyes which pixels aresuccessive along the Y direction. Moreover, the specification section 11a, for example, obtains the coordinates (Y1 and Y2) of the two cornersof the mouth which coordinates concern the Y direction, and thecoordinate (Y3) pertaining to the position of the lowermost end of themouth shown in FIG. 8 which coordinate (Y3) concerns the Y direction(see FIG. 8).

After the processing at Step S13, the face image processing section 11performs the correction processing of an eye (Step S14).

Here, the correction processing of an eye will be described withreference to FIG. 5.

The comparison section 11 b compares the sizes of the two eyes to eachother on the basis of the information pertaining to the sizes of the twoeyes which information has been obtained by the processing at Step S13(Step S41). Then, the comparison section 11 b specifies the larger eye(Step S42). To put it concretely, the comparison section 11 b, forexample, compares the numbers of the pixels of the respective pupils ofthe two eyes with each other which pixels are successive along the Ydirection. Then, the comparison section 11 b specifies the eye in whichthe number of the pixels thereof is relatively larger.

Moreover, the comparison section 11 b obtains the ratio of the sizes ofthe two eyes (Step S43). The comparison section 11 b, for example,calculates the ratio of the numbers of the pixels of the respectivepupils of the two eyes which pixels are successive along the Ydirection, and specifies the calculated ratio as that of the sizes ofthe two eyes.

After the processing at Step S43, in order that the size of the one eyein the width direction thereof which eye has been specified as thelarger one by the processing at Step S42 may be almost the same as thatof the other eye in the width direction, the correction section 11 ccorrects the other eye, that is, the smaller eye, by enlarging the eyeon the basis of the ratio of the sizes of the two eyes calculated by thecomparison section 11 b by the processing at Step S43 (Step S44) (seeFIG. 8). With that, the correction processing of an eye ends.

After the correction processing of an eye, the face image processingsection 11 performs the correction processing of a corner of the mouth(Step S15).

Here, the correction processing of a corner of the mouth will bedescribed with reference to FIG. 6.

The comparison section 11 b calculates each of the positions of the twocorners of the mouth on the basis of the information pertaining to thepositions of the two corners of the mouth obtained by the processing atStep S13 (Step S51). Then, the comparison section 11 b compares thepositions of the two corners of the mouth to each other (Step S52), andspecifies the corner of the mouth the position of which is higher thanthat of the other corner of the mouth (Step S53).

After the processing at Step S53, the correction section 11 c uses thecorner specified as the one situated at the higher position by theprocessing at Step S53 to correct the position of the other corner ofthe mouth so as to align the position with that of the higher corner(Step S54) (see FIG. 8). With that, the correction processing of acorner of the mouth ends.

After the correction processing of a corner of the mouth, the displaycontrol section 7 displays the correction results by the correctionsection 11 c (Step S16). Moreover, the display control section 7performs a saving propriety confirming display for selecting whether tosave the displayed correction results or not (Step S17). After that, theCPU 10 waits for an input operation of the determination button 9 c orthe cancel button 9 d of the operation input section 9 to the savingpropriety confirming display displayed by the processing at Step S17(Step S18: NO).

When an input operation of the determination button 9 c or the cancelbutton 9 d of the operation input section 9 has been performed to thesaving propriety confirming display by the processing at Step S17 (StepS18: YES), the correction execution determining section 11 e judgeswhether an input operation to the determination button 9 c of theoperation input section 9 has been performed or not (Step S19). If theinput operation of the determination button 9 c of the operation inputsection 9 has been performed (Step S19: YES), the correction executiondetermining section 11 e outputs the image data corresponding to thedisplayed correction results to the JPEG compression section of theimage processing section 11 (Step S20). On the other hand, if it isjudged that no input operations of the determination button 9 c of theoperation input section 9 have been performed in the processing at StepS19 (Step S19: NO), that is, if the cancel button 9 d has been operated,the correction execution determining section 11 e discards the displayedcorrection results (Step S21) and outputs the image data of the faceimage before the corrections to the JPEG compression section of theimage processing section 11. After the processing at Step S20 or S21,the correction processing ends.

After the end of the correction processing, the recording medium 6records the image data coded by the JPEG compression section of theimage processing section 11 (Step S4). With that, the processing ends.

Although the flow chart of FIG. 2 describes the correction processing ofa face image obtained by the imaging thereof by the processing at StepS1, the correction processing can be performed not only at the time ofimaging the face image. For example, the correction processing can beperformed to the image data of a face image already stored in therecording medium 6.

Moreover, although the flow chart of FIG. 2 automatically performs thecorrection processing to the image data of an imaged face image, theconfiguration enabling the previous setting of whether to automaticallyperform a correction of a face image immediately after the imagingthereof or not may be adopted. For example, the following method can becited in which the setting of whether to automatically perform acorrection of a face image immediately after the imaging thereof or notis determined before the imaging thereof on the basis of an inputoperation of the determination button 9 c or the cancel button 9 d ofthe operation input section 9, or the like. The determined setting ofwhether to automatically perform a correction of a face imageimmediately after the imaging thereof or not may be recorded into astorage device, such as the recording medium 6, to be held therein.

As described above, according to the imaging apparatus 100 of thepresent embodiment, the lens section 1, the electronic imaging section2, and the imaging control section 3 obtain a face image; thespecification section 11 a of the face image processing section 11specifies a set of a plurality of corresponding areas; and thecorrection section 11 c of the face image processing section 11 uses oneof the specified plurality of areas of the face image as a reference togenerate a face image in which the other area is corrected.

That is, because the one area of the face image constitutes the faceimage in the natural state thereof subjected to no corrections, acorrection result of the other area subjected to the correction can bemade to be one giving a natural impression by using the one area as areference to correct the other area. As described above, the imagingapparatus 100 can perform more natural image processing to a face image.

Moreover, the specification section 11 a specifies each of a set of aplurality of corresponding face constituting parts (for example eyes andthe corners of a mouth), and the correction section 11 c uses one of theplurality of specified face constituting parts as a reference to correctthe other face constituting part.

Hereby, it becomes possible to correct a face image by correcting a faceconstituting part without correcting the whole area of the face image.Consequently, the processing load of the correction can be reduced ascompared to that in the case of correcting one area of the face image.Moreover, a local correction of a face image can be performed bycorrecting a face constituting part of the face image.

Moreover, the specification section 11 a specifies each of the eyes eachexisting in each of the two areas of a face image as each of a pluralityof face constituting parts, and the correction section 11 c uses one ofthe specified two eyes as a reference to correct the other eye.

Hereby, because the eye (the other eye) of the face image can becorrected on the basis of the one, the correction result of the eye canbe made to be the one giving a natural impression.

Moreover, the correction reference determining section 11 d sets alarger eye of two eyes as the one eye used as the correction reference,and the correction section 11 c corrects the size of the other eye onthe basis of the size of the one eye. Thereby, the size of therelatively smaller other eye can be aligned with the size of therelatively larger one. Generally, a face image having large eyes givesan expressive and cheerful impression, and consequently a face image canbeautifully be seen as one giving an expressive and cheerful impressionas compared to that in the case of performing no corrections bycorrecting the smaller eye on the basis of the larger eye.

Moreover, the specification section 11 a specifies each of the cornersof a mouth each existing in each of the two areas of a face image as thetwo face constituting parts, and the correction section 11 c uses one ofthe specified two corners of the mouth as a reference to correct theother corner.

Hereby, by using the one corner of the mouth as a reference, the cornerof the mouth (the other corner of the mouth) of the face image can becorrected, and consequently the correction result of the corner of themouth can be made to be one giving a natural impression.

Moreover, the correction reference determining section 11 d determinesone of a set of a plurality of corresponding areas to be used as acorrection reference of a correction by the correction section 11 c onthe basis of a predetermined condition.

Hereby, one area to be used as a correction reference can easily bedetermined.

For example, an eye and a corner of a mouth to be used as correctionreferences can automatically be determined by using the predeterminedconditions “the eye specified as the larger one by the comparisonsection 11 b is used as the one eye of a correction reference” and “thecorner of a mouth specified by the comparison section 11 b as the onesituated at a higher position is used as the one corner of a correctionreference,” both of which have been shown in the above embodiment. Inthis case, because the correction processing of a face image can beautomatized, the correction processing of the face image can beperformed without imposing troublesome operations upon a user.

Moreover, the display control section 7 makes the display section 8display a correction result by the correction section 11 c of the faceimage processing section 11. Then, the correction execution determiningsection 11 e determines whether to save a correction by the correctionsection 11 c or not on the basis of the content of a selection inputreceived with the determination button 9 c and the cancel button 9 d ofthe operation input section 9.

Hereby, a user can perform the decision making of whether to perform acorrection of a face image or not after ascertaining the correctionresult of the face image.

The configuration of the imaging apparatus 100 shown in the aboveembodiment is only one example of those of the imaging apparatus 100 and100A, and the configurations are not limited to the above-shown one. Thepresent invention is not limited to the above embodiment, and variousimprovements and changes of the design of the embodiment may beperformed without departing from the scope and sprit of the presentinvention.

<Modification>

FIG. 9 is a diagram showing a modified imaging apparatus 100A.

The modified operation input section 9 thereof further includes aselection button 9 e for a designation input of a face constituting partto which a correction is performed.

The correction section 11 c determines the face constituting part to becorrected on the basis of the content of a designation input with theselection button 9 e of the operation input section 9.

To put it concretely, the selection button 9 e of the operation inputsection 9 receives a designation input of the kind of a faceconstituting part to which a correction by the correction section 11 cis performed among two or more sets of a plurality of corresponding faceconstituting parts, that is, two or more kinds of sets of a plurality ofcorresponding face constituting parts (for example, eyes and the cornersof a mouth) specified by the specification section 11 a. At this time,the display section 8 may display the face image to be corrected toenable a user to ascertain the face image to determine the kind of theface constituting part to which a correction is performed.

The correction processing of the modification will be described withreference to the flow charts of FIGS. 10 and 11.

First, the specification section 11 a performs area specifyingprocessing (Step S61).

After the area specifying processing, the specification section 11 aspecifies face constituting parts (for example two eyes and the twocorners of a mouth) each existing in each of the two areas (Step S62).The specification section 11 a specifies the eyes and the corners of themouth on the basis of, for example, differences of pixel values,brightness and the like between the eyes and the mouth and theirperipheral areas of the face image. Moreover, the specification section11 a obtains the coordinates indicating the information pertaining tothe sizes of the two eyes, the information pertaining to the position ofthe lowermost end of the mouth, and the information pertaining to thepositions of the two corners of the mouth (Step S63). The specificationsection 11 a, for example, obtains the numbers of the pixels of therespective pupils of the two eyes which pixels are successive along theY direction as the information pertaining to the sizes of the two eyes.Moreover, the specification section 11 a, for example, obtains thecoordinate (Y3) of the position of the lowermost end of the mouth shownin FIG. 8 which coordinate (Y3) pertains to the Y direction and thecoordinates (Y1 and Y2) of the two corners of the mouth whichcoordinates (Y1 and Y2) pertain to the Y direction as the informationpertaining to the position of the lowermost end of the mouth and theinformation pertaining to the positions of the corners of the mouth.

After the processing at Step S63, the face image processing section 11waits for the selection button 9 e of the operation input section 9 toreceive a designation input of a kind of the face constituting part towhich a correction by the correction section 11 c is performed betweenthe two kinds of face constituting parts (eyes and the corners of amouth) specified by the specification section 11 a (Step S64: NO). Whenthe selection button 9 e receives a designation input of the kind of theface constituting part to which a correction by the correction section11 c is performed (Step S64: YES), the CPU 10 judges whether an eye isdesignated as a correction object or not (Step S65). If the eye isdesignated as the correction object (Step S65: YES), the face imageprocessing section 11 performs the correction processing of an eye (StepS66).

After the correction processing of an eye or if it is judged that an eyeis not designated as a correction object at Step S65 (Step S65: NO), theCPU 10 judgers whether a corner of a mouth is designated as a correctionobject or not (Step S67). If it is judged that the corner of a mouth isdesignated as a correction object (Step S67: YES), the face imageprocessing section 11 performs the correction processing of a corner ofa mouth (Step S68).

After the correction processing of a corner of a mouth or if it isjudged that a corner of a mouth is not designated as a correction objectat Step S67 (Step S67: NO), the CPU 10 judges whether a correction ofany face constituting part by the correction section 11 c has beenperformed or not (Step S69). If no corrections have been performed toany face constituting parts by the correction section 11 c (Step S69:NO), the correction execution determining section 11 e outputs the imagedata of a face image to the JPEG compression section of the imageprocessing section 11 and ends the correction processing.

If it is judged that a correction of any face constituting part by thecorrection section 11 c has been performed at Step S69 (Step S69: YES),the display control section 7 displays the correction result by thecorrection section 11 c (Step S70). Moreover, the display controlsection 7 performs a saving propriety confirming display for selectingwhether to save the displayed correction result or not (Step S71). Afterthat, the CPU 10 waits for an input operation of the determinationbutton 9 c or the cancel button 9 d of the operation input section 9 tothe saving propriety confirming display displayed by the processing atStep S71 (Step S72: NO).

When the input operation of the determination button 9 c or the cancelbutton 9 d of the operation input section 9 has been performed to thesaving propriety confirming display displayed by the processing at StepS71 (Step S72: YES), the correction execution determining section 11 ejudges whether the input operation has been performed to thedetermination button 9 c of the operation input section 9 or not (StepS73). If it has been judged that the input operation has been performedto the determination button 9 c of the operation input section 9 (StepS73: YES), the correction execution determining section 11 e outputs theimage data corresponding to the displayed correction result to the JPEGcompression section of the image processing section 11 (Step S74). Onthe other hand, if it has been judged that no input operations have beenperformed to the determination button 9 c of the operation input section9 at Step S73 (Step S73: NO), that is, if the cancel button 9 d has beenoperated, the correction execution determining section 11 e discards thedisplayed correction result (Step S75) and outputs the image data of theface image before the correction to the JPEG compression section of theimage processing section 11. After the processing at Step S74 or S75,the correction processing ends.

As described above, according to the modification, the face constitutingpart to which a correction by the correction section 11 c is performedamong the two or more kinds of the face constituting parts specified bythe specification section 11 a is determined according to the content ofan input operation by a user to the selection button 9 e.

Hereby, a correction can be performed to a face constituting part that auser desires, and a correction result of the face image that a userdesires can easily be obtained.

As another modification, the correction reference determining section 11d may have the function of determining an area of a correction referencein accordance with a selection result of a user. In this case, forexample, the display control section 7 makes the display section 8display a face image obtained by the electronic imaging section 2 andthe imaging control section 3. Then, a designation input of a userconcerning which one of the two areas (for example two eyes or the twocorners of a mouth) of the face image displayed in the display section 8is used as the one area of a correction reference is received on thebasis of designation input content to the selection button 9 e or thelike of the operation input section 9. The correction referencedetermining section 11 d determines one area of a correction referencein accordance with the received designation input content.

One eye of a correction reference is not limited to larger one, but maybe smaller one. Moreover, one corner of a mouth of a correctionreference is not limited to the one situated at a higher position, butmay be one situated at a lower position.

If the correction reference determining section 11 d has the function ofdetermining an area of a correction reference in accordance with aselection result of a user, the information pertaining to one area of acorrection reference selected by a user maybe recorded by the recordingmedium 6. At this time, the modification may be configured as follows: auser name and a selection result of the user may be associated with eachother, and when the user uses the imaging apparatus 100 or 100A afterthat, the user can read out the selection result associated with theuser name only by designating or selecting the user name.

By providing the function of determining a correction reference inaccordance with a user's selection result to the correction referencedetermining section 11 d, the user can designate a favorite faceconstituting part as a reference among the two areas of a face image andthe face constituting parts existing in the respective two areas.Hereby, because the other face constituting part is corrected on thebasis of the user's favorite face constituting part, the corrected faceimage becomes closer to the user's favorite. That is, a correctionresult of a face image that a user desires can easily be obtained.

The image processing apparatus of the present invention is not limitedto the imaging apparatus 100 and 100A. For example, the image processingapparatus may be configured to obtain the image data of a face imagefrom external equipment to perform a correction of the face image.

Moreover, although eyes and the corners of a mouth have been exemplifiedas face constituting parts in the above embodiment and modifications,face constituting parts (for example eyebrows) other than the eyes andthe corners of the mouth may be specified to be corrected.

Moreover, a correction of a face image is not limited to the correctionsof the size of an eye and the position of a corner of a mouth. Forexample, the shape or the color of a face constituting part, such as aneye or a mouth, or the shape or the color of one of the two areas of aface image may be corrected. Moreover, three or more areas may be set.For example, the generation of a face image in which the skin color ofthe area of a forehead is used as a reference to correct the skin colorsof other areas (for example both cheeks) of the face image can be cited.

Moreover, the method of specifying an eye or a corner of a mouth is notlimited to the described content of the embodiment and themodifications. For example, the method of extracting the contour of aface constituting part to be specified can be cited. Moreover, only thepixels in an extracted contour maybe treated as a face constitutingpart, and a square area including a contour may be treated as a faceconstituting part.

Moreover, a face image may be divided to the left and the right, and,for example, the face image on the left side may be used as a referenceto correct the face image on the right side. Or, a reversed image of aface image on either the left or the right may be generated, and acorrection may be performed so as to generate bilaterally symmetric faceimages.

Moreover, the processing of dividing a face image into two parts may beomitted, and the specification section 11 a may be configured to specifya set of two corresponding face constituting parts.

Moreover, although the correction results by the correction section 11 care displayed in the above embodiment and the modifications, the displayprocessing may be omitted and the correction results may automaticallybe saved.

Moreover, as for a correction of a face image, the correction section 11c may generate a plurality of correction results different from eachother and may display each of the plurality of correction results. Then,one of the plurality of correction results that has been selected by auser may be saved. As an example of showing a plurality of correctionresults, the following case can be cited: showing both of a correctionresult of the case of performing a correction by expanding a smaller eyeon the basis of a larger eye and a correction result of the case ofperforming a correction by reducing a larger eye on the basis of asmaller eye.

By generating a plurality of correction results different from eachother to provide options of a correction result to be saved, it becomeseasy to reflect a correction result that is more favorite one for auser, and a corrected face image becomes closer to a user's favorite.That is, a correction result of a face image that a user desires canmore easily be obtained.

The extent of a correction may be changed according to the inclinationof a face image. For example, if a face image is imaged from the rightdirection toward the front of the face of a subject, the area and thesize of the face constituting part on the right side of the face imageare larger in comparison with those on the left side, and a correctionfor balancing the face image in the state of imaging the face image fromthe right direction may accordingly be performed.

In addition, although the above embodiment is configured to be realizedby a drive by the face image processing section 11 under the control ofthe CPU 10, the configuration is not limited to the above one, but theconfiguration may be realized by the execution of a predeterminedprogram and the like by the CPU 10.

That is, a program memory (the illustration of which is omitted) forstoring a program previously stores a program including an obtainmentprocessing routine, an area specifying processing routine, a faceconstituting part specification processing routine, and a correctionprocessing routine. Then, the CPU 10 may be functioned to obtain a faceimage by the obtainment processing routine. Moreover, the CPU 10 may befunctioned to specify a set of a plurality of corresponding areas in theobtained face image by the area specifying processing routine. Moreover,the CPU 10 may be functioned to specify a set of a plurality ofcorresponding face constituting parts in the obtained face image by theface constituting part specification processing routine. Moreover, theCPU 10 may be functioned to generate a face image in which one of aplurality of corresponding areas specified by an area specifying sectionor one of a plurality of specified corresponding face constituting partsin a face image is used as a reference and the other of the plurality ofcorresponding areas or the plurality of corresponding face constitutingparts is corrected by the correction processing routine.

The entire disclosure of Japanese Patent Application No. 2010-139026filed on Jun. 18, 2010 including description, claims, drawings, andabstract are incorporated herein by reference in its entirety.

Although various exemplary embodiments have been shown and described,the invention is not limited to the embodiments shown. Therefore, thescope of the invention is intended to be limited solely by the scope ofthe claims that follow.

1. An image processing apparatus, comprising: an obtainment section toobtain a face image; an area specifying section to specify a set of aplurality of corresponding areas in the face image obtained by theobtainment section; and a correction section to generate a face image inwhich one area of the plurality of corresponding areas of the face imageis used as a reference to correct another area.
 2. The image processingapparatus according to claim 1, further comprising: a determinationsection to determine the one area of the reference of a correctionperformed by the correction section among the plurality of correspondingareas based on a predetermined condition.
 3. The image processingapparatus according to claim 2, further comprising: a correctionreference designation receiving section to receive a designation inputof the one area of the reference of the correction performed by thecorrection section among the plurality of corresponding areas specifiedby the area specifying section, wherein the determination sectiondetermines the one area of the reference of the correction performed bythe correction section based on content of the designation inputreceived by the correction reference designation receiving section. 4.The image processing apparatus according to claim 1, wherein the areaspecifying section further specifies a set of a plurality ofcorresponding face constituting parts in the face image, and thecorrection section generates a face image in which one face constitutingpart of the plurality of corresponding face constituting parts of theface image specified by the area specifying section is used as areference to correct another face constituting part.
 5. The imageprocessing apparatus according to claim 4, wherein the area specifyingsection specifies both eyes existing in the face image as the set of theplurality of corresponding face constituting parts, and the correctionsection uses one eye of both the eyes specified by the area specifyingsection as the reference to correct another eye.
 6. The image processingapparatus according to claim 2, wherein the area specifying sectionspecifies both eyes existing in the face image as the set of theplurality of corresponding areas; the determination section determinesone eye of both the eyes specified by the area specifying section as thereference of a correction; and the correction section uses a size of theeye determined by the determination section as the reference to correcta size of another eye.
 7. The image processing apparatus according toclaim 4, wherein the area specifying section specifies two corners of amouth existing in the face image as the set of the plurality ofcorresponding face constituting parts, and the correction section usesone corner of both the corners of the mouth specified by the areaspecifying section as the reference to correct another corner of themouth.
 8. The image processing apparatus according to claim 4, furthercomprising: a correction object designation receiving section to receivea designation input of a face constituting part of a correctionperformed by the correction section among two or more sets of theplurality of corresponding face constituting parts specified by the areaspecifying section, wherein the correction section determines the faceconstituting part based on content of the designation input received bythe correction object designation receiving section.
 9. An imageprocessing apparatus, comprising: an obtainment section to obtain a faceimage; a face constituting part specifying section to specify a set of aplurality of corresponding face constituting parts in the face imageobtained by the obtainment section; and a correction section to generatea face image in which one face constituting part of the plurality ofcorresponding face constituting parts of the face image is used as areference to correct another face constituting part.
 10. An imageprocessing method for making an image processing apparatus execute thepieces of processing comprising: obtaining a face image; specifying aset of a plurality of corresponding areas in the obtained face image;and generating a face image in which one are of the plurality ofspecified corresponding areas of the face image is used as a referenceto correct another area.
 11. An image processing method for making animage processing apparatus execute the pieces of processing comprising:obtaining a face image; specifying a set of a plurality of correspondingface constituting parts in the obtained face image; and generating aface image in which one face constituting part of the plurality ofspecified corresponding face constituting parts of the face image isused as a reference to correct another face constituting part.
 12. Arecording medium recording a program for making a computer installed inan image processing apparatus execute the functions comprising: anobtainment section to obtain a face image; an area specifying section tospecify a set of a plurality of corresponding areas in the face imageobtained by the obtainment section; and a correction section to generatea face image in which one area of the plurality of corresponding areasof the face image specified by the area specifying section is used as areference to correct another area.
 13. A recording medium recording aprogram for making a computer installed in an image processing apparatusexecute the functions comprising: an obtainment section to obtain a faceimage; a face constituting part specifying section to specify a set of aplurality of corresponding face constituting parts in the face imageobtained by the obtainment section; and a correction section to generatea face image in which one face constituting part of the plurality ofcorresponding face constituting parts of the face image is used as areference to correct another face constituting part.